Lines of Latitude & Longitude

Concepts & Themes:

This week’s assignment will encompass the following concepts covered in Class 2 lecture demo & lab:

Visual Analogies for Latitude & Longitude:

A mapped GCS coordinate

Latitude as rungs on ladder

Longitude as telephone poles

Class 2 Readings:

Assignment II Map: Large Scale Mapping of NYC social responses to Covid-19

Preamble:

  • As Social Distancing requirements were enacted in New York City and New York State, Spring 2020, an interesting - maybe expected - response surfaced, captured by NYC’s 311 Service Requests. As reported, complaints about others Not social distancing spiked - so much so, that a new descriptor was developed in the 311 dataset: Descriptor = Social Distancing. Amazingly, these complaints number 60,000+ since the end of March when the descriptor was incorporated into the 311 dataset.

  • In this assignment, we will map the 311 data, conduct a spatial operation then visualize the results both in cartographic and chart outputs. Often GIS output will feature a combination of aspects, attributes or dimensions of a dataset. A recent example of this approach to data visualization from National Geographic - Monumental Undertaking shows first the temporal dimension of Confederate statues, followed by their spatial distribution:

Example of data presentation - chart + map

  • The following mapping assignment utilizes the spatial distribution of Social Distancing complaints, alongside an aggregate count per NYC borough. By visualizing both the disaggreate data points and aggregate count, the final product allows a meaningful count in addtion to spatial distribution alone.

  • To Start, first acquire the data in .csv format, then map it against the 5 boroughs in NYC. The Latitude and Longitude spatial attributes in the dataset will be used to transform the tabular dataset into a spatial dataset with geometry - .csv to .shp. Along the way, a total complaint count per borough will be determined, and finally a rate of complaint per borough. A map product is then developed that shows the pattern of the data and charts the relative complaint count per borough.

  • To start, the following filer query and rate formula will be utilzed in the assignment:

    • To filter the 311 data:`Descriptor is Social Distancing`
    • In order to determine a square mile per borough:shape_area * 0.00000003587006428 = square mile

Data Acquisition:

NYC OpenData

  • Scroll down to the 311 Dataset link and click:

311 Service Requests

  • Note that there are multiple 311 Service Request instances. Make sure to select the Dataset, not Data Lens instance:

311 Dataset

  • Once selected, continue with View Data button selection:

View Data

  • Apply a Filter based on query Descriptor is Social Distancing:

Applying Data Filter

  • The resulting subselection would be 62,000 + records with the Social Distancing Descriptor attribute:

Record Results

  • Download the subselection .csv to a folder directory for Part II. Accept the default naming convention of the .csv:

Download Record Results

  • If desired, open the .csv in Excel and navigate to right-end of the file. Here you will see Latitude and Longitude. These two columns will be utilized to read in the locational data for the creating of geometry (points) within QGIS:

Locational Data

  • Finally, download the Class2 map data folder. Here you will find the borough geometry, and a backup .csv of the 311 data as of 01/29/21. Only use this .csv as testing data, not data for the assignment per se:

Class 2 Assignment Data

Data Organization within QGIS:

  • Open QGIS and point the Data Source Manager to the .csv service requests utilizing the Delimited Text data source option. Make sure to populate the tool as seen below and rename the layer simply as 311_SR:

Delimited Text - QGIS

  • Next, load the borough boundary shapefile, and position the service requests atop the borough boundary and symbolize so that the the points are approximately 0.25 size in black, and the boroughs are a light neutral color of your choice:

NYC Borough Geography

Note: There will be some stray points outside NYC area. These are incorrect lat/lon data points wherein the lat/lon was recorded incorrectly in the 311 dataset. Disregard these points. Also now is the time to begin to save your .qgs file on a regular basis in order to avoid duplicate work in the event of a software crash.

  • You may notice that your data ‘shape’ is horizontally elongated compared to the image above. This is due to the fact that the data is cast within wgs84 coordinate system. We can change that easily so that the ‘shape’ is much more accurate. To do so, we will utilize the map projection designed for the NYC area (State Plane system). First click on the CRS button, lower right of Map Canvas. Once done, search for 3104 which is a short code for the correct map projection for NYC. Then click Add at bottom of tool window. The shape of the data will automatically update. This is a type of ‘on the fly’ projection; we are changing the Map Canvas coordinates, but we are leaving the actual .prj file on each of the data sources unchanged. This is a ‘quick and dirty’ yet efficient cartographic kludge (hack):

Location of CRS reference - lower right of Map Canvas

CRS Filter

Note: you may run into a CRS projection issue with the polygon feature for NYC boroughs. IF your features disappear from your Map Canvas as you perform an ‘on the fly’ projection for the data frame, the following video will solve this problem:

  • Addressing the Missing CRS delimma:

Lecture 1

  • Next, we want know how many complaints exist per borough expressed in the following structured question:

    • Which borough has the most ‘Social Distancing’ public complaint reports due to Covid-19 distancing mandates in New York City and State ?
    • Which borough has the highest rate of public complaint reports based on square area (square miles) ?

Note: for this second question, perhaps a more insightful query would be rate of public complaints per 10K population. Certainly we can answer this question also, but we would need population data which we will cover in depth in Class 4 - Thematic Mapping.

  • Utilize the Count points in polygon overlay operation to get a total count of complaint records per borough. This tool is available from the Main Menu:

  • Vector > Analysis Tools > Count Points in Polygon

Imgur

  • Populate the tool with the polygon and points ordered as follows:

Imgur

  • This operation will result in a new temporary layer COUNT. Immediately export this new layer as a .shp into your working directly, entitled 311_borough:

Imgur Imgur

Note: see that the new .shp now contains a field column entitled NUMPOINTS in integer format (int8) - this is the count per borough.

  • Next, calculate the complaints per borough and the rate of complaints per borough in the attribute table of the new 311_borough.shp. To do this navigate to the Toolbar and choose the Field Calculator tool:

Imgur

  • Make sure 311_borough is active in the layers panel. Create a new field sq.mile and populate with the following formula, using Fields and Values > shape_area dropdown:

    • shape_area * 0.00000003587006428

Imgur

Note: This formula takes the current areal quanity in sq. feet and converts it to a sq. mile - a unit that is more appropriate to the size of the borough polygons.

  • To complete, save the attribute addition; Toggle Editing to the off position and Save:

Imgur

  • With the number of complaints per borough (NUMPOINTS) and the sq. miles of each borough (sq.mile), a rate normalized to sq. area can be achieved. Again, open the Field Calculator and use the same process as above to Create a new field, but this time name the field rate.sq.mi and use the following formula:

  • "NUMPOINTS" / "sq.mile"

  • Again, save the attribute addition; Toggle Editing to the off position and Save.

  • Open the attribute table and view the results. Its clear the rate of complaints per sq. mile in Manhattan is highest, even as its total count is lower than Brooklyn. The influence of the outer neighborhoods in Brooklyn is likely bringing down the rate in Brooklyn as a whole. In other words, the most populated areas of Brooklyn are likely on par with the rate of complaints per sq. mile as Manhattan.

Imgur

Note: if you are having problems with the Field Calculator, getting to this stage of the assignment, the following video will walk through each step and may be helpful to get back on track:

  • Creating Sq Miles & Rates with QGIS Field Calculator:

Lecture 1

Visualizing the analysis results - cartographic and graphing design

  • With the analysis steps complete, we move along to cartographic design and output. We can augment the map design with a graphing feature to good effect. The following components will be featured in this final stage:

    • Utilize QGIS Plugin architecture.
    • Produce cartographic output; augment with graph design.
    • Incorporate HTML elements into the Map Layout.

  • To start, make sure the .qgs files is saved.

Note: as there will be 60K points on this map, consider ramping down opacity and keep small symbol size. For the example map, opacity of points has been set at approx. 75%, and the size 0.25.

  • Next, in order to incorporate graphing components into map layouts, we’ll use the Data Plotly plugin. This is one of the new features within QGIS 3.x. To install plugins, utilize the Plugins Main Menu:

Imgur

  • Search for DataPlotly. When the plugin loads, choose to Install plugin:

Imgur

  • Once installed, the following icon will be placed on the main Toolbar menu. Go ahead and click atop the icon to open the DataPlotly toolset:

Imgur

  • The first icon on the left (paintbrush and bucket) is the initial setup for graph. Here we choose a bar graph, the layer and the x and y axis. Importantly, the bar color, stroke color and width can be determined:

Imgur

  • Next, move to the icon below (gears) to set the title, axis labels. Once complete, click Create Plot in the lower section of the tool window. This creates a preview of the graph:

Imgur

  • Finally, click the last icon to access the HTML expression of the chart. This code can be highlighted, copied and then placed into a text editor (do not use MS Word. In the Class 2 lab, text editors will be discussed. There are many free, accessible editors).

Imgur

  • With the code complete and copied safely to a text editor, Create a new layout instance, and Save Project from within Map Layout. Include the following elements:

    • Main Map elements - 311 Social Distance Request Locations + NYC Boroughs
    • Map/Graph author tag (your name) + data source
    • Map legend
    • Embedded Data Plotly graph

  • The first three elements were covered generally in Class 1, Assignment 1 so you should have some familiarity with their creation and design. The fourth element - the Data Plotly graph - is a bit more challenging, but with some practice both valuable and efficient.

  • To start the graph, click the Add a New HTML to the Layout icon at left and draw out a element box on the Layout Canvas:

Imgur

  • The result is a new empty element both on the map and in the right items panel. Copy the HTML code from the text editor into the HTML source section with the Source button activated. Click Return and the chart will appear on the Layout Canvas:

Imgur

Tips:

  • You will often want to toogle the stacking order of the various elements on the map page. For instance, negative space surrounding the map elements may partially obscure the graph and vice versa. You can toogle one atop the other in a desired order from the Actions Toolbar:

Imgur

  • The font style is difficult to change within Data Plotly. Given this constraint, you can either use the same font as that of Data Plotly - Open Sans - or you can find a complimentary font style for your title, legend items, ect.

  • Download Open Sans font - free - Open Sans Font

  • Pair with complimentary font styles

  • Review the Assignment Example before finalizing your assignment. You are encouraged to experiment and choose a different approach than the example, but do indeed include all elements found in the example. Upload your PDF export to CANVAS assignment 2 submission location.

Video Guides:

A Missing CRS

Area & Rates

Assignment References:

Additional References:

Additional Class 2 Spatial realated data/readings/articles:

To follow, a short tutorial on Degree|Minute|Second conversion to Decimal lat/lon expression:

latlonDMS

Further, you can make the DMS>Decimal Degrees and vice versa on a point-by-point basis using an online converter like the following:

Additional ‘out of the box’, downloadable Lat/Lon datasets: